Predicting Memory Activity Using Spatial Correlation

The memory wall continues to pose a performance bottleneck for computer systems—studies show that modern servers spend up to two-thirds of execution time stalled on memory accesses. Although recent trends forecast growth in processor clock frequencies to be minimal, improvements to memory access latencies are correspondingly slow. Traditional approaches, such as large on-chip caches, hardware multithreading, and out-of-order processing, demonstrate some success at mitigating the impact of high memory latencies, but offer little hope of completely overcoming the memory wall. Prefetching/streaming techniques have been proposed for predicting and eliminating misses in desktop, scientific, and engineering applications, but are less effective across commercial workloads, which exhibit data dependent and irregular memory behaviors. Though complex, commercial server applications nevertheless organize their data in a structured manner and at large granularity. In this thesis, we explore spatial correlation of access patterns that span page-sized regions of memory. We develop mechanisms for accurately observing and predicting repetitive spatial layouts, which lead us to propose Spatial Memory Streaming (SMS), a hardware prefetcher that exploits spatial correlation to predict cache misses in server workloads. Although SMS is practical and effective, it loses significant opportunity for performance improvement because it predicts spatial layouts in isolation. To overcome this fundamental limitation, we adapt temporal correlation to predict order across spatial layouts. We exploit the repetitive temporal characteristics of spatial accesses in Spatio-Temporal Memory Streaming (STeMS), a hardware prefetching technique that combines spatial and temporal predictions into a single total predicted miss sequence. Across our suite of commercial workloads, we demonstrate that STeMS achieves equal or better prediction coverage and application speedup than either temporal or spatial memory streaming alone.